Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.
For simplicity, let’s try it out directly:
Edge Computing on Raspberry Pi with Movidius Neural Compute Stick
Finally, I’m going to talk about Raspberry Pi a bit. It has been discussed so widely that NOTHING special should be further talked from me.
To write something about Raspberry Pi is to say GOOD BYE to my Raspberry Pi 3B, and WELCOME Raspberry Pi 4 at the same time. Our target today is to build an AI edge computing end as the following video:
1. Raspberry Pi 3B vs. Raspberry Pi 3B+
Before everythig starts, let’s carry out a simple comparison between Raspberry Pi 3B and Raspberry Pi 3B+:
Raspberry Pi 3B vs. Raspberry Pi 3B+
2. Movidius Neural Compute Stick on Raspberry Pi 3B
Then, we just follow the following 2 blogs Run NCS Applications on Raspberry Pi and Adding AI to the Raspberry Pi with the Movidius Neural Compute Stick to utilize these 2 outdated products:
| Raspberry Pi 3B | Intel Movidius Neural Compute Stick 1 |
|---|---|
 |
 |
Intel Movidius Neural Compute Stick 1 is NOT listed on Intel’s official website any more. But github support for Intel Movidius Neural Compute Stick 1 can be found at https://github.com/movidius/ncsdk.
2.1 Raspberry Pi 3B’s Wifi Module Does NOT Work Properly
2.1.1 Wifi NOT Working
I met a bug that I failed to find the solution forever. Three related forum issues are enumerated as follows, and you are welcome to carry out further investigation.
- Raspberry Pi 3 Cannot Connect to Wi-Fi
- (UPDATE) Drivers for TL-WN725N V2 - 3.6.11+ -> 4.xx.xx+
- Default WiFi adapter name (e.g. wlan0)
Anyway, ip -c address gives out the following ERROR message:
1 | ➜ ~ ip -c address |
2.1.2 Use a Wifi USB Dongle
An arbitrary 802.11 N Wifi 2.4G USB dongle is able to activate Wifi for Raspberry Pi 3B in my case. BTW, mine is EDUP EP-N8508GS.
1 | ➜ ~ ip -c address |
To bring up Wifi is just to plug in/out the Wifi USB dongle.
2.2 Prerequisite Installation
2.2.1 NCSDK
We FIRST need to have ncsdk installed. Yup, here, as described in Run NCS Applications on Raspberry Pi, we carry out the installation directly under folder ...../ncsdk/api/src.
1 | ➜ src git:(ncsdk2) ✗ make -j4 |
2.2.2 Caffe
2.2.2.1 Prerequisite Packages
Prerequisite:
For simplicity, you can grab the above three packages directly from Raspbian’s repository. However, it seems the default boost from Raspbian’s repository does NOT support python3 but ONLY python2. Therefore, in my case, I built boost 1.71, flann 1.9.1 and OpenCV 4.1.1 from sractch from source.
2.2.2.2 Cross-compile For flann
The reason why we need to cross-compile flann is because of the limited memory of Raspberry Pi 3B.
In my case, I’m using crosstool-NG and following blog: RPi Linaro GCC Compilation to carry out my cross-compiling. Details about cross-compiling can be found in my next blog Build Toolchain Using Crosstool-NG
Additional Issue: BCM2837 is detected as BCM2835
1 | ➜ ~ cat /proc/cpuinfo |
This issue seems to be an OLD topic. More details about the development history of Raspberry Pi can be found on Wikipedia.
2.2.2.3 Build Caffe From Source
Then, we build Caffe 1 from source.
2.3 Test NCS
Now, let’s test the installation of NCS, by the example hello_ncs_py from ncappzoo.
1 | ➜ hello_ncs_py git:(ncsdk2) python hello_ncs.py |
2.2.3 Test Simple Image Classification
Build Toolchain Using Crosstool-NG
Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.
In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:
1. Installation
How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:
1 | pwd |
And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.
2. Configuration
Under any directory that you want to save your .config file, we can configure our target cross compiler.
1 | ct-ng menuconfig |
According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:
-
Paths and misc options:
- Try features marked as EXPERIMENTAL: ticked
- Prefix directory: input the full path where you want to save the built toolchains
- Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.
-
Target options:
- Target Architecture: arm
- Default instruction set mode: arm
- Use EABI: ticked
- append
hfto the tuple (EXPERIMENTAL): ticked - Endianness: Little endian
- Bitness: 32-bit
- Emit assembly for CPU: cortex-a53
- Use specific FPU: vfp
- Floating point: hardware (FPU)
-
Toolchain options:
- Tuple’s vendor string: rpi
-
Operating System
- Target OS: linux
- Version of Linux: 4.20.8. Currently, I flashed Raspbian Buster with desktop and recommended software, 2019-07-10 onto my Raspberry Pi 3B, it comes with kernel of version 4.19.66, which is NOT in the list. That’s why I decided to select 4.20.8 and try my luck.
1
2➜ ~ uname -a
Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux -
Binary utilities:
- Binary format: ELF
- Version of binutils: 2.31.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19➜ ~ apt show binutils
Package: binutils
Version: 2.31.1-16+rpi1
Priority: optional
Section: devel
Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 95.2 kB
Provides: binutils-gold, elf-binutils
Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
Suggests: binutils-doc (>= 2.31.1-16+rpi1)
Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
Homepage: https://www.gnu.org/software/binutils/
Download-Size: 56.9 kB
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU assembler, linker and binary utilities
The programs in this package are used to assemble, link and manipulate
binary and object files. They may be used in conjunction with a compiler
and various libraries to build programs. -
C-library:
- C library: glibc
- Version of glibc: 2.28
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25➜ ~ apt show libc-bin
Package: libc-bin
Version: 2.28-10+rpi1
Priority: required
Essential: yes
Section: libs
Source: glibc
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Installed-Size: 3,015 kB
Depends: libc6 (>> 2.28), libc6 (<< 2.29)
Recommends: manpages
Homepage: https://www.gnu.org/software/libc/libc.html
Download-Size: 657 kB
APT-Manual-Installed: yes
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C Library: Binaries
This package contains utility programs related to the GNU C Library.
.
* catchsegv: catch segmentation faults in programs
* getconf: query system configuration variables
* getent: get entries from administrative databases
* iconv, iconvconfig: convert between character encodings
* ldd, ldconfig: print/configure shared library dependencies
* locale, localedef: show/generate locale definitions
* tzselect, zdump, zic: select/dump/compile time zones -
C compiler:
- Show gcc versions from: GNU
- Version of gcc: 8.3.0
- gcc extra config: --with-float=hard
- Link libstdc++ statically into the gcc binary: tick
- C++: tick
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20➜ ~ apt show gcc
Package: gcc
Version: 4:8.3.0-1+rpi2
Priority: optional
Section: devel
Source: gcc-defaults (1.181+rpi2)
Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
Installed-Size: 46.1 kB
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Download-Size: 5,200 B
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.
After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.
3. Build
Please remember to:
unset LD_LIBRARY_PATHbefore building. Otherwise, you’ll meet some ERROR messages.mkdir ~/srcbefore building. Otherwise, whenever you tried to rerunct-ng build, you’ll have to download ALL required packages from scratch.
1 | ct-ng build |
This process may take a while. I’m going to sleep tonight. Continue tomorrow…
Alright, let’s continue today. Glad to know it’s built successfully.
Let’s FIRST take a look at what’s under the current folder.
1 | ls |
And then, let’s take a look at what’s built under the specified destination folder.
1 | ls ~/....../CrossCompile/RPi |
Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.
1 | arm-rpi-linux-gnueabihf-gcc --version |
Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG
4. Compile/Build a Package with Generated Toolchain
For most of the packages nowadays, they are
- either supported by make:
./configure -> make -> make install - or supported by CMake:
mkdir build -> cd build -> ccmake ../ -> make -> make install
How to use our generated toolchain to compile/build our target packages?
- For the FIRST option, you can follow the crosstool-NG Using the toolchain.
- For the SECOND option, you are welcome to follow CMake Toolchains
Today, we’re going to take the package flann as an example, which are to be built with CMake.
4.1 CMakeLists.txt Modification
4.1.1 CMake Toolchains - Cross Compiling for Linux
By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).
1 | cmake_minimum_required(VERSION 2.6) |
- CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
- CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
- tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder
${tools}/bin.
4.1.2 Ignore hdf5
In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.
1 | #find_hdf5() |
4.2 Cross Compile
Now, let’s start cross-compiling flann.
1 | mkdir build |
and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:
1 | BUILD_CUDA_LIB *OFF |
The LAST step before make is to modify some of the parameters accordingly, in my case:
- BUILD_PYTHON_BINDINGS: ON -> OFF
- CMAKE_CXX_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
- CMAKE_C_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
- CMAKE_VERBOSE_MAKEFILE: OFF -> ON
- CMAKE_INSTALL_PREFIX:
/usr/local/-> under which directory you want to install, which can be IGNORED for now
Now, press c, you’ll see
1 | CMake Warning at CMakeLists.txt:99 (message): |
A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.
Finally, it’s the time for us to cross build flann.
1 | make -j8 |
VERY Important:
- If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory..
- Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.
Finally, you’ll see flann has been successfully cross compiled as:
1 | ....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]': |
You can now:
make installto install the built/generated libraries installed under CMAKE_INSTALL_PREFIX- copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.
BTW: do NOT forget to install the header files.
5. Additional Issues
Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.
- Multilib caveats from Official Notes on specific toolchain features
- fatal error: gnu/stubs-hard.h: No such file or directory from crosstool-NG
- fatal error: gnu/stubs-hard.h: No such file or directory from gcc-cross-compiler
- Build error: “features.h: No such file or directory” from risc-v tools
- crosstool-NG risc-v linux multilib issue
- raspberrypi toolchain issue
Alright, that’s all for today. Let me go to bed. Good bye…
Stereo Vision on VCSBC nano Z-RH-2 - PART I
Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.
1. VCSBC nano Z-RH-2
1.1 Hardware
We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:
| Front | Back | Connector |
|---|---|---|
 |
 |
 |
More detailed specifications, please refer to Vision Components’s official website.
1.2 Software
After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.
1 | ssh user@192.168.1.79 |
Currently, VC provides Linux Kernel 3.14.79.
1 | user@VC-Z:~$ uname -a |
And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.
1 | user@VC-Z:~$ cat /proc/cpuinfo |
2. Stereo Vision
Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.
Hmmmmmmmm… It’s better I keep my code in dark???
2.1 Classical Image Pairs
Let’s try out the stereo vision on some .pgm image pairs FIRST.
1 | user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm |
My GOD… It’s UNBELIEVABLY SLOW.
| aloe_left | aloe_right |
|---|---|
 |
 |
| aloe_left Stereo | aloe_right Stereo |
 |
 |
2.2 Live Video Pairs
The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.
2.2.1 imageCaptureTest
1 | ./imageCaptureTest |
2.2.2 imageCaptFPS
1 | 03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS |
The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.
2.3 Stream/Display From Host Computer
Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.
2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)
Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.
- Eclipse
- CDT: install by Check for Updates with p2 software repository: http://download.eclipse.org/tools/cdt/releases/9.8
- Direct Remote C++ Debugging: installed from Eclipse Marketplace
2.3.2 vcimgnetclient.py
Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.
Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.
2.3.2.1 PyGTK to PyGTK3
1) 2to3
1 | 2to3 vcimgnetclient.py |
2) pygi-convert
Please refer to General Porting Tips
1 | ./pygi-convert.sh vcimgnetclient.py |
3) Try Running
1 | python vcimgnetclient.py |
4)
For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:
vbox.pack_start(self.menu_bar, False, False)
to
vbox.pack_start(self.menu_bar, False, False, 0)
Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.
2.3.3 Longer Vision’s vcimgnetclient_qt.py
2.3.3.1 Server
1 | 22:42:22[root@VC-Z] /home/user |
2.3.3.2 Client
Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:
Kaggle Competition - APTOS 2019 Blindness Detection
Oh-my-god, kaggle competition about diabetis… Hmmm, I have something to do these days… It’s also a GOOD chance for me to learn some medical terms… In this blog, we’re going to try out this kaggle competition - APTOS 2019 Blindness Detection.
1. Pre-processing Diabetic Retinopathy Images
It’s NOT hard for us to locate APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, which seems to me a professional snippet of code preprocessing diabetic retinopathy images. Let’s just try it out directly.
1.1 Implementation
The implementation is cited directly from APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, with trivial modifications. You are welcome to download my snippet of code here.